Sparse Coding in the Neocortex

Sparse coding has been proposed as a guiding principle in neural representations of sensory input, particularly in the visual system. Because sparse codes are defined as representations with low activity ratios—i.e., at any given time a small proportion of neurons are active—they are sometimes proposed as a means to help conserve metabolic costs. Although we accept that such metabolic costs play a role in evolutionary selection, we also argue that sparse coding offers a useful solution to the problem of representing natural data because such a scheme allows the system to take advantage of the sparse structure of the sensory environment. Given the highly regular and sparse structure of natural data (e.g., natural scenes), a variety of recent studies suggest that the sparse coding in the cortex is an adaptation that has evolved to offer an efficient strategy for storing and building associations with natural inputs. However, highly selective